SETUP

The code requires hardware with GPUs.

Run the following command:

    conda env create -f MNIST_env.yml

RUNNING THE EXPERIMENT

After the environment is created and activated, run the following command:

    python main.py --RunSGD $RunSGD --eps $eps --RunSGDwReg $RunSGDwReg --reg_eps $reg_eps --lr $lr --decay_rate $decay_rate --weight_decay $weight_decay --bs $bs --lambda1 $lambda1 --torchseed $torchseed

You can change the number after each argument (e.g., $RunSGD, $eps, etc.) as needed.

Description of arguments:

--RunSGD: runs the original SGD if set as 1, does not run if set as 0.

--eps: number of epochs for the original SGD.

--RunSGDwReg: runs SGD with one regularizer if set as 1, does not run if set as 0.

--reg_eps: number of epochs for SGD with one explicit regularizer.

--lr: starting learning rate for both algorithms.

--decay_rate: decay rate for the learning rates.

--weight_decay: strength for the weight decay regularizer.

--bs: batch size for both algorithms.

--lambda1: strength of the regularizer (used with SGD with one regularizer).

--torchseed: random seed for the torch package.


OUTPUT

The code will output the results for the two algorithms. For each algorithm, the training loss, test loss, and test accuracy are reported for every training epoch.